Method and system to detect substrate placement accuracy

ABSTRACT

A method and system for measuring the alignment between a substrate and a platform upon which it is disposed by using image processing algorithms are described herein. These algorithms automate the detection of edges of a microscope slide and the platform in a digital image. A reference line pattern in an image of the platform can be used to detect platform edges based on a computed location of the reference line pattern in the image.

RELATED APPLICATION DATA

This is a continuation of International Patent Application No.PCT/EP2017/079447, filed Nov. 16, 2017, which claims the benefit andpriority of U.S. Provisional Patent Application No. 62/423,859, filedNov. 18, 2016, both of which applications are incorporated by referenceherein.

FIELD OF THE INVENTION

A method and system are disclosed for detecting substrate placementaccuracy, and in particular for detecting placement accuracy ofmicroscope slides in automated slide staining instruments and slideimaging instruments.

BACKGROUND

Inconsistent placement of a substrate onto a platform for automatedtreatment can lead to inconsistent processing of the substrate,particularly where application of reagents to the substrate depends uponthe position of the substrate in order to ensure that reagents areapplied to the correct location(s), and that they remain there or areconveyed away appropriately. For example, when microscope slides bearingbiological samples are treated for microscopic analysis, improperpositioning of the microscope slide on or in a slide staining platform(such as a heating element or a treatment chamber) may lead to fluiddispense and removal errors, or movement of fluid away from the sample(such as by wicking). If reagents are not properly removed between thesteps of a staining protocol, or reagent is not in contact with thesample during a particular step, the entire staining protocol may bealtered and render the slide unreadable by a pathologist. Anothercontext where substrate placement is important is where microscopeslides are placed on a platform (such as a stage) for imaging of thestained biological samples thereon. In either case, improper positioningcan lead to inconsistent or invalid analysis results.

SUMMARY

An automated method and system are disclosed for measuring the alignmentbetween a substrate and a platform upon which the substrate is disposed.In one embodiment, a transparent glass microscope slide bearing abiological sample is placed on a processing platform for one or moreslide treatment operations.

For example, based on imaged features of the slide and images offeatures of a slide processing platform supporting the slide, a slidemisalignment condition can be detected, by comparing features extractedfrom the image to a pre-determined cutoff values If the conditionexceeds or falls below the cutoff, as the case may be, an error isdetected. The detected error can be communicated to a user and/or theslide can be automatically repositioned and re-measured for alignment.The alignment detection and slide repositioning and re-measurementprocess can be repeated one or more times, for example, up to apre-determined number of times, before a failure is indicated and theprocess/system is halted for inspection and/or repair.

In a more particular embodiment, at least a portion of an image of theslide processing platform is obtained through a transparent substratesuch as a glass or plastic microscope slide. In an even more particularembodiment, the slide processing platform includes at least one featureof known length and/or orientation that can be imaged through thesubstrate and upon which the slide alignment condition can becalculated.

Any feature or combination of features described herein are includedwithin the scope of the method for detecting slide placement accuracyfor medical device instruments provided that the features included inany such combination are not mutually inconsistent as will be apparentfrom the context, this specification, and the knowledge of one ofordinary skill in the art. Additional advantages and aspects of themethod for detecting slide placement accuracy for medical deviceinstruments are apparent in the following detailed description andclaims.

In some embodiments, the platform is comprised of a horizontal edge anda vertical edge and the microscope slide is comprised of a slidehorizontal edge and a slide vertical edge that connect to form aright-angled corner. Further comprising the platform is a reference linepattern joining the vertical and horizontal edges of the platform suchthat the right-angled corner of the microscope slide extends beyond thereference line pattern. From an image analysis perspective, potentiallighting variations in platform images may lead to intensity variations.A simple threshold-based method cannot locate the aforementioned pairsof edges. The disclosed image analysis based solution correctlyidentifies the reference line pattern, while capturing the slide edges.Detecting the reference line pattern is helpful to the remainder of theimage analysis solution, where the horizontal and vertical linesrepresenting the platform edges can subsequently be accuratelydetermined based on the initially computed location of the referenceline segment. If a misalignment is detected, then the user is notified.Experimental results show that the detected horizontal and verticaledges correspond very closely to the lines picked up by visualinspection.

Also disclosed is a system for detecting misalignment of a substrate(such as a microscope slide) placed on a platform (such as a heating orcooling platform) and a slide placement mechanism for automaticallyplacing a slide onto the platform.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the method for detecting slide placementaccuracy for medical device instruments will become apparent from aconsideration of the following detailed description presented inconnection with the accompanying drawings in which:

FIG. 1A shows a slide placement image.

FIG. 1B shows the slide placement image zoomed in to demonstrate theangle, θ, that the reference line pattern makes with the positive x-axisof the x-y coordinate plane of the image.

FIG. 2 is an image of the inverse grayscale image where the diagonalline segment and other salient horizontal and vertical lines show up aswhite, i.e. the corresponding pixel magnitudes (intensities) are highwhile they show up black in the original RGB image (i.e. the grayscalevalues are near zero).

FIG. 3 shows the detection (A) of the diagonal line segment according toCase 1 as described herein.

FIG. 4 shows a cropped image of the diagonal line segment (A) of Case 1as described herein, where the algorithm crops the image to verify thatthe diagonal line segment was correctly chosen.

FIG. 5 shows an image of an exemplary cost function matrix where therows indicate the different starting locations of the best row index forthe left-most location of the diagonal line segment, while the columnsindicate the different locations of the best column index for theleft-most location The plot is represented in “colormap jet” in MATLAB,where blue color denotes lower pixel magnitudes and the red colordenotes higher values.

FIG. 6 shows transposed images of vertical line segments according to adisclosed embodiment, wherein in panel (A) is shown the raw image of thetransposed line segment and in panel (B) a candidate line is shownsuperimposed on the transposed line segment image.

FIG. 7 shows transposed images of the sloping vertical line segmentsaccording to a disclosed embodiment, wherein in panel (A) is shown theraw image of the transposed line segment and in panel (B) a candidateline is shown superimposed on the transposed line segment image.

FIG. 8 shows the detection and highlighting of a horizontal line segmentof a platform (B) and a horizontal line segment of a microscope slide(B), wherein the shortest distance between the segment could bedetermined from the left edge of the image where the line segmentsterminate.

FIG. 9A shows an embodiment of a slide detection system.

FIG. 9B shows an embodiment of a slide detection and alignment system.

DETAILED DESCRIPTION

Further features and aspects of the disclosed system and method aredescribed in the exemplary embodiments that follow.

Referring now to FIGS. 1A and 1B, in one aspect, a method is disclosedof detecting a misalignment condition between a microscope slide (102)and a platform (104) upon which the microscope slide (102) is disposed.In one embodiment, the method includes identifying, in an image thatincludes at least a portion of the microscope slide (102) and at least aportion of the platform (104), a platform feature (110); (b) identifyingin the image, a first edge of the platform (106) and a first edge of themicroscope slide (112), wherein identification of the first edge of theplatform (106) in the image is aided by identification of a firstreference point (118) of the platform feature (110); (c) calculating avalue of a first distance between the first edge of the platform (106)and the first edge of the microscope slide (112) in the image, whereinif the value of the first distance lies outside a first pre-determinedrange of values, a misalignment condition between the microscope slide(102) and the platform (104) is detected.

In a particular embodiment, the method further includes (a) identifying,in the image, a second edge (108) of the platform (104) and a secondedge (114) of the microscope slide (102), wherein identification of thesecond edge (108) of the platform in the image is aided byidentification of a second reference point (120) of the platform feature(110); and, (b) calculating a value of a second distance that is ashortest distance between the second edge (108) of the platform (104)and the second edge (114) of the microscope slide, wherein if the valueof the first distance lies outside the first pre-determined range ofvalues or if the value of the second distance lies outside a secondpre-determined range of values, a misalignment condition between themicroscope slide (102) and the platform (104) is detected.

In another particular embodiment, the method further includes convertingthe image to a grayscale image (for example, see FIGS. 2-4, and 6-8) toaid in detection of one or more of the platform feature (110), the firstedge of the platform (106), the second edge of the platform (108), thefirst edge of the microscope slide (112), and the second edge of themicroscope slide (114).

In yet another particular embodiment, the platform feature (110) islocated in an image frame such that it forms a known angle (see FIG. 1B)with an x-axis of an x-y coordinate plane of the image. In still otherparticular embodiments, the platform feature (110) is of a known length.In even other particular embodiments, the platform feature (110) isimaged through the microscope slide.

In more particular embodiments, identifying one or more of the platformfeature (110), the first edge of the platform (106), the second edge ofthe platform (108), the first edge of the microscope slide (112), andthe second edge of the microscope slide (114) includes selecting from apotential set of line segments a line segment to represent the one ormore of the platform feature (110), the first edge of the platform(106), the second edge of the platform (108), the first edge of themicroscope slide (112) and the second edge of the microscope slide(114). In even more particular embodiments, selecting includes selectinga potential line segment based on a cost function (see, for example,FIG. 5).

In still other more particular embodiments, at least one of the firstedge of the platform (106), the second edge of the platform (108), thefirst edge of the microscope slide (112), and the second edge of themicroscope slide (114) appears as a vertical line in an image andidentifying further comprises transposing at least a portion of theimage prior to selecting a line segment to represent the one or more ofthe platform feature (110)(see FIGS. 6 and 7), the first edge of theplatform (106), the second edge of the platform (108), the first edge ofthe microscope slide (112) and the second edge of the microscope slide(114).

In another embodiment, the disclosed method further includesrepositioning the microscope slide on the platform in response todetection of a misalignment condition. In further the disclosed methodis repeated following a repositioning action to detect if themisalignment condition is resolved.

In another aspect, and with respect to FIGS. 9A and 9B, a system isdisclosed for determining a slide misalignment condition between amicroscope slide (102) and a platform (104). In one embodiment, thesystem includes (a) a camera (200) disposed above the microscope slide(102) and the platform (104) such that at least a portion of themicroscope slide (102) and at least a portion of the platform (104) arepositioned in a field of view of the camera; and (b) a processor (202),wherein the processor is configured to operate according to instructionsstored in a memory (201) to control the camera to obtain an image and toperform any of the disclosed embodiments of the method to detect theslide misalignment condition.

In one embodiment, the system further includes a slide alignment device(204), wherein the memory (201) further stores instructions that causethe processor (202) to control the slide alignment device to position orreposition the microscope slide (102) on the platform (104) eitherinitially, or in response to a detected misalignment condition,respectively. The processor can be further caused by instruction storedin the memory to repeatedly reposition the microscope slide one or moretimes, such as up to a pre-determined number of times, before a failurecondition is indicated.

In particular embodiments, the platform (104) can be a heating and/orcooling platform of an automated slide staining device. In otherparticular embodiments the platform (104) includes a platform feature(110) detectable in the field of view of the camera (202), and theplatform feature (110) can be of known length and/or appears in a knownorientation in the image obtained by the camera.

In other particular embodiments, the microscope slide is transparent andthe platform feature (110) can be imaged through the microscope slide.

In another embodiment, the slide alignment device (204) is controlled bythe processor (202) according to instructions stored in the memory (201)that cause the slide alignment device to place the slide onto theplatform (104) such that a right-angled corner (116) of the microscopeslide (102) extends over the platform feature (110).

EXAMPLES

In one embodiment, the original RGB image is smoothed using a 3×3 medianfilter followed by a 5×5 Gaussian kernel to aide in removing small localartifacts and discontinuities in the image, which may have an effect ondetecting the misalignment. The algorithm then creates an inversegrayscale of the original RGB image to produce the digital image used bythe algorithm (where the inverse grayscale image may be defined as(1−double(grayscale)/255)). This digital image is divided into fourquadrants: a top left quadrant, a top right quadrant, a bottom leftquadrant, and a bottom right quadrant. Considering the top rightquadrant of the digital image, the platform (104) has a horizontal edge(106) and a vertical edge (108) and the microscope slide (102) has aslide horizontal edge (112) and a slide vertical edge (114) that connectto form a right-angled corner (116). In a grayscale version of theoriginal image, the pair of horizontal edges (106,112) and verticaledges (108,114) appear darker relative to their immediate background. Aspreviously mentioned, the digital image is an inverse grayscale imagewhere the pair of horizontal (106,112) and vertical edges (108,114)appear near white. These near white pixies have magnitudes that aresignificantly higher than surrounding pixels (pixels comprising theimmediate background).

The aforementioned pairs of horizontal and vertical lines in the digitalimage may not be perfectly horizontal or perfectly vertical to oneanother. In some cases, these vertical and horizontal lines may be veryfaint, thus if the main focus is detecting these two pairs of lines,robustness issues will arise. The disclosed method instead focuses ondetecting a reference line pattern (110) common to all slide placementimages. This reference line pattern (110) is located in the top rightquadrant of the digital image and joins the vertical edge (108) of theplatform (104) to the horizontal edge (106), such that the right-angledcorner (116) of the microscope slide (102) extends beyond the referenceline pattern (110). Three cases are considered: (i) the case in whichthe reference line pattern is a diagonal line segment creating a 135°angle with the positive x-axis of the x-y coordinate plane, (ii) thecase in which the reference line pattern is a diagonal whose angle withthe positive x-axis is only approximately known, and (iii) the case inwhich the reference line pattern may represent any pattern (e.g. a linesegment of any angle with respect to the x-axis, or a curve) as long asthe length and the parametric form of the line pattern is known.

To illustrate the method, the image processing algorithm was implementedin MATLAB. However, it should be clear to one of ordinary skill in theart that execution of the algorithm is not limited to MATLAB, and it maybe implemented in other programing environments such as C++, and thelike.

Case 1: The reference line pattern is a diagonal line segment creating a135° angle with respect to the positive x-axis.

As mentioned previously, the algorithm identifies the location of thediagonal line segment (110) joining the horizontal edge (106) and thevertical edge (108) of the platform (104), where a length of thediagonal line segment (110) is known. Also, an angle, θ, between thediagonal line segment (110) and the positive x-axis (of the x-ycoordinate plane) measures 135 degrees. In one embodiment, to identifythe location of the diagonal line segment (110), a first set ofpotential line segments is generated. Potential line segments are linesegments that reflect the known features of the diagonal line segment(110), i.e., the angle between each potential line segment and thepositive x-axis measures 135 degrees and the length of each potentialline segment is the known length of the diagonal line segment (110). Theset of potential line segments are located in the top right quadrant ofthe digital image (100).

To identify the potential line segment that is most likely the locationof the diagonal line segment (110), a first cost function is calculatedfor each. Further, each potential line segment comprises a set of pixels(each pixel in the set of pixels having a magnitude). In an exemplaryemobodiment, (r,c) may denote the location of a potential line segment,where r is the row index and c is the column index in the top rightquadrant of the digital image. It is assumed that the length, L, of thediagonal line segment (110) is known. Thus, with a top left location at(r,c), a search window is created from row=r to row=r+L and fromcolumn=c to column=c+L. The diagonal line segment (110), definitivelylocated within this window, will contain all the high valued pixelswhile a standard deviation will be low (high or low relative to thecorresponding values of the remaining pixels in the window). For eachpotential line segment, considering the inverse grayscale image, thefirst cost function is represented as:Cost=(mean of the magnitude of the pixels comprising the diagonal linesegment and immediate off-diagonal pixels)/(standard deviation of thepixels comprising the diagonal line segment and immediate off-diagonalpixels+0.001).

Immediate off-diagonal pixels, (that is, pixels that are either onepixel above or one pixel below each pixel comprising a potential linesegment), are also considered as in some cases the diagonal line segment(110) may be blurred. Considering more pixels makes the average/standarddeviation based computation more robust. Additionally, the 0.001 in thedenominator ensures that division by zero will not occur even if all ofthe pixels included in the computation have the same magnitude. Thediagonal line segment (110) is comprised of pixels having magnitudesapproximately equal in value and having magnitudes higher than those ofsurrounding pixels. Therefore, the set of pixels comprising an optimalpotential line segment have an average magnitude higher than themagnitudes of surrounding pixels and a near-zero standard deviation.Thus, selecting the potential line segment that yields a highest valueof the first cost function identifies the location of the diagonal linesegment (110).

Identifying the location of the diagonal line segment (110) provides tworeference points in the digital image effective for locating thevertical edge (108) of the platform (104) and the slide vertical edge(114) (and subsequently the horizontal edge (106) of the platform (104)and the slide horizontal edge (112)). These two reference points are aleft-most location (118) and a right-most location (120) of the diagonalline segment (110).

In another embodiment, the next step of the algorithm is locating thevertical edge (108) of the platform (104). A transposed version of thetop right quadrant of the digital image (100) is obtained and a secondset of potential line segments are generated such that each potentialline segment has a slope between −0.1 to 0.1. This second set ofpotential line segments are located within a first window of thetransposed version of the top right quadrant of the digital image.Experimentally, the vertical edge (108) of the platform (104) was foundto lie typically within 30 pixels to the right of the rightmost locationof the diagonal line segment (110). Thus, the first window comprisescolumns of thirty pixels beginning at the right-most location (120) ofthe diagonal line segment (110) and extending upwards and rows of pixelsbeginning from the right-most location (120) of the diagonal linesegment (110) and extending (to the right) to an end of the transposedversion of the top right quadrant of the digital image (100). Furtherembodiments feature a second cost function calculated for each potentialline segment. Each potential line segment comprises a set of pixels,wherein each pixel of the set of pixels has a magnitude greater than acutoff magnitude. The second cost function is calculated as:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment yielding a highest value of the second costfunction identifies the location of the vertical edge (108) of theplatform (104).

Another embodiment next identifies the location of the slide verticaledge (114). Locating the slide vertical edge (114) may comprisegenerating a third set of potential line segments having a slope withina range of −0.2 to 0.2 (since the slide vertical edge (114) is typicallymore slanted than the vertical edge (108) of the platform (104)). Thisset of potential line segments are also located within the transposedversion of the top right quadrant of the digital image (100). Morespecifically, the third set of potential line segments are locatedwithin a second window comprising columns of pixels beginning 10 pixelsabove the right most location (120) of the diagonal line segment (110)and extending 250 pixels above the right-most location (120) of thediagonal line segment (110) and all rows of pixels in the transposedversion of the top right quadrant of the digital image. A third costfunction may then be calculated for each potential line segment, whereeach potential line segment comprises a set of pixels each pixel havinga magnitude greater than the cutoff magnitude. The third cost functionis calculated as:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment yielding the highest value of the third costfunction identifies the location of the slide vertical edge (114).

A transposed version of the image was obtained when locating thevertical edge (108) of the platform (104) and the slide vertical edge(114) because the vertical edges become horizontal lines in thetransposed image and hence, the algorithm looks for lines with slopes inthe range [0.1, −0.1]. It is easier to define a window of 0.1 around anexpected slope when searching for lines with a slope close to zero. Ifsearching for lines with slopes close to infinity (vertical lines), thenthe window is difficult to define.

In some embodiments, locating the horizontal edge (106) of the platform(104) in the original (non-transposed, inverse grayscale) digital imageis next. A fourth set of potential line segments, each having a slopewithin a range of [−0.1, 0.1] may be generated. This set of potentialline segments are located within a third window. The third window maycomprise rows of pixels within 30 pixels above the left-most location(118) of the diagonal line segment (110), and all columns of pixels inthe top right quadrant of the original digital image (100). A fourthcost function for each potential line segment may then be calculated,where each potential line segment comprises a set of pixels having amagnitude greater than the cutoff magnitude. Similar to the second andthird cost function, the fourth cost function is calculated as follows:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment yielding the highest value of the fourth costfunction identifies the location of the horizontal edge (106) of theplatform (104).

Further locating the slide horizontal edge (112) in the original(non-transposed, inverse grayscale) digital image follows. A fifth setof potential line segments, each having a slope within a range of [−0.2,0.2] may be generated (since the slide horizontal edge (114) istypically more slanted than the horizontal edge (106) of the platform(104)). The fifth set of potential line segments are located within afourth window. The fourth window may comprise rows within 33 pixels to250 pixels above the left-most location (118) of the diagonal linesegment (110) and all columns in the top right quadrant of the originaldigital image (100). A fifth cost function for each potential linesegment may then be calculated. Each potential line segment comprises aset of pixels, where each pixel of the set of pixels has a magnitudegreater than the cutoff magnitude. The fifth cost function is calculatedas follows:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment that yields the highest value of the fifthcost function is selected, thereby identifying the slide horizontal edge(112).

After identifying the locations of the pair of horizontal edges(106,112) and the pair of vertical edges (108,114) of the of theplatform (104) and the microscope slide (102), a first and seconddistance is calculated. The value of the first distance is the shortesthorizontal distance between the vertical edge (108) of the platform(104) and the slide vertical edge (114). The value of the seconddistance is the shortest vertical distance between the horizontal edge(106) of the platform (104) and the slide horizontal edge (112). If thevalue of the first distance is greater than a first threshold maximumvalue or smaller than a first threshold minimum value then amisalignment between the microscope slide (102) and the platform (104)is detected. A misalignment is also detected If the value of the seconddistance is greater than a second threshold maximum value or smallerthan a second threshold minimum value.

In one embodiment, the algorithm exports pixel data of each digitalimage to a pre-populated spreadsheet where calculations for thepass/fail criteria are made as seen in Table 1 below.

TABLE 1 Non-limiting Example of Slide Placement Test Results a b c dHorizontal Vertical Horizontal Vertical Horizontal Horizontal VerticalVertical e Slide over-hang over-hang over-hang over-hang conditioncondition condition condition Pass/ ID (pixels) (pixels) (mm) (mm)(>=0.6 mm) (<=1.9 mm) (>=0.3 mm) (<=1.0 mm) Fail 1 86.00 53.50 1.52 0.940.92 0.38 0.64 0.06 Pass 2 86.00 141.50 1.52 2.50 0.92 0.38 2.20 −1.50Fail 3 79.85 55.79 1.41 0.98 0.81 0.49 0.68 0.02 Pass 4 40.16 80.47 0.711.42 0.11 1.19 1.12 −0.42 Fail 5 90.34 57.00 1.59 1.01 0.99 0.31 0.71−0.01 Fail 6 41.50 29.50 0.73 0.52 0.13 1.17 0.22 0.48 Pass 7 102.0019.32 1.80 0.34 1.20 0.10 0.04 0.66 Pass 8 33.80 49.32 0.60 0.87 0.001.30 0.57 0.13 Pass 9 107.91 21.15 1.90 0.37 1.30 0.00 0.07 0.63 Pass 10102.00 56.50 1.80 1.00 1.20 0.10 0.70 0.00 Pass

Calculation:

Heater width=22.5 mm (1275 pixels)

Horizontal over-hang threshold: >=0.60 mm AND<=1.90 mm

Vertical over-hang threshold: >=0.30 mm AND<=1.00 mm

The following is a description of what is calculated in each of thecolumns of the spreadsheet of Table 1:

(b) Slide Over-hang Values in Pixels: These are slide over-hang values(expressed in pixels) imported directly from the algorithm program (e.g.the MATLAB test script in the present case).

(c) Pixel-to-Millimeter Conversion: Calculation to convert the pixelmeasurements in (b) to millimeters.

(d) Slide Over-Hang Condition Calculation: Calculates the amount ofslide over-hang based on the requirements.

(e) Pass/Fail Criteria: Calculates and outputs the Pass/Fail indicationbased on the condition calculations.

In some embodiments, when locating the diagonal line segment (110), thepotential line segments representing potential locations of the diagonalline segment (110) are located within a fifth window. In the fifthwindow, a left-most location (118) of the diagonal line segment (110) islocated within rows 25% to 50% from a top row of the digital image andwithin columns 55% to 83% from a left-most column of the top rightquadrant of the digital image (100).

Further, one or more predetermined values of the cutoff magnitude isused to narrow the set of potential line segments. For robustnesspurposes, the algorithm may test different cutoffs when significantpixels (those contributing to a potential line segment) are considered.Cutoff values tested were in the range [0.8, 0.6, 0.5, 0.4, 0.2]. Foreach cutoff, pixels comprising a potential line segment must be largerthan the given cutoff, for those satisfying this constraint, the costfunction is calculated (this holds for each of the five cost functions).In other embodiments, the algorithm considers potential line segmentsthat yield the 10 highest cost functions (for each of the five costfunctions) and sorts them in decreasing order. If the mean of the pixelscomprising a potential line segment is less than 0.8 times the cutoff,then the algorithm discards it. The cost function yielding the highestvalue is then selected.

For example, when locating the vertical edge (108) of the platform(104), using a cutoff of 0.6, the cost, slope with respect to thex-axis, and y-intercept of the top 10 potential line segments were found(as seen in Table 2).

TABLE 2 COST SLOPE Y-INTERCEPT 24.4503 −0.0045 38.6222 24.1824 −0.001236.2973 24.1295 0.0000 35.0000 23.9320 −0.0050 38.4523 23.7867 −0.001135.2555 23.2985 −0.0056 39.3322 22.6123 0.0029 32.5234 22.2790 −0.005140.0169 21.4830 0.0023 31.9241 20.9726 0.0027 31.6578

The potential line segment with a slope of −0.0045 and a y-intercept of38.622 was selected as the diagonal line segment (110). Similarly, thecost, slope and y-intercept of the top two potential line segments forthe slide vertical edge (114) were found using a cutoff of 0.8, to be:

COST SLOPE Y-INTERCEPT 8.0503 0.0296 18.3964 7.9779 0.0299 18.0351

The potential line segment yielding the highest cost function isselected as the slide vertical edge (114). Once the two vertical edgesare identified, the shortest horizontal distance between these two linesis calculated considering the image portion below the bottom-mostlocation of the diagonal line segment (110). The distance between thetwo most prominent vertical lines is 40.16 (34) as seen in Table 1.

The cost, slope with respect to the x-axis, and y-intercept of the top10 potential line segments were found (as seen in Table 3) in thedetection of the horizontal edge (106) of the platform (104) as well.Based on these results, a potential line segment with slope=0.0030 andy-intercpet=28.6870 was chosen. After experimenting with theaforementioned cutoff values, a cutoff of 0.6 acquired the highest costfunctions and the potential line segment selected had a cost of 4.7281,a slope of −0.0313 and a y-intercept of 168.2500. Referring to Table 1,for these horizontal lines, the minimum vertical distance was found tobe 80.47 (12).

TABLE 3 COST SLOPE Y-INTERCEPT 17.3246 0.0030 28.6870 17.2981 0.003327.5299 17.1581 0.0019 28.3270 17.1325 0.0016 29.9822 17.0916 0.002529.4887 17.0587 0.0036 28.7925 17.0020 0.0037 29.2829 16.9065 0.000031.0000 16.8707 0.0056 28.5778 16.7518 0.0086 24.6658

Case 2: The reference line pattern is a diagonal line segment creatingan angle with respect to the positive x-axis that is approximatelyknown.

In an embodiment, detecting a misalignment between the microscope slide(102) and the platform upon which the microscope slide (102) is disposedcomprises first locating the diagonal line segment (110) that joins thehorizontal edge (106) and the vertical edge (108) of the platform (104).Here, the diagonal line segment (110) has a length L and a referenceangle θ, where θ is the angle between the diagonal line segment (110)and the positive x-axis. The algorithm first generates a plurality ofpotential line segments each having a length equal to L and a referenceangle equal to θ. Each potential line segment is defined by a centerpixel. Next, the center pixel of each potential line segment may besuperimposed on each pixel of a plurality of pixels comprising the topright quadrant of the inverse grayscale digital image. A first costfunction is calculated for each potential line segment for each pixel inthe top right quadrant of the digital image (100). Each potential linesegment is comprised of a set of pixels, where each pixel in the set ofpixels has a magnitude. In some embodiments, the first cost function maydefined as follows:cost=(mean of the magnitude of the pixels comprising the diagonal linesegment and immediate off-diagonal pixels)/(standard deviation of thediagonal and immediate off-diagonal pixels+0.001).

The immediate off-diagonal pixels are pixels either one pixel above orone pixel below each potential line segment and are also considered asin some cases the diagonal line segment (110) may be blurred.Considering more pixels makes the average/standard deviation basedcomputation more robust. Additionally, the 0.001 in the denominatorensures that division by zero will not occur even if all of the pixelsincluded in the computation have the same magnitude. The diagonal linesegment (110) is comprised of pixels having magnitudes approximatelyequal in value and having magnitudes higher than those of surroundingpixels. Therefore the set of pixels comprising an optimal potential linesegment have an average magnitude higher than the magnitudes ofsurrounding pixels and a near-zero standard deviation. Thus, selectingthe potential line segment that yields a highest value of the first costfunction identifies the location of the diagonal line segment (110).

Identifying the location of the diagonal line segment (110) provides tworeference points in the digital image effective for locating thevertical edge (108) of the platform (104) and the slide vertical edge(114) (and subsequently the horizontal edge (106) of the platform (104)and the slide horizontal edge (112)). These two reference points are aleft-most location (118) and a right-most location (120) of the diagonalline segment (110).

In another embodiment, the next step of the algorithm is locating thevertical edge (108) of the platform (104). A transposed version of thetop right quadrant of the digital image (100) is obtained and a firstset of potential line segments are generated such that each potentialline segment has a slope between −0.1 to 0.1. This set of potential linesegments are located within a first window of the transposed version ofthe top right quadrant of the digital image. The first window maycomprise columns of pixels beginning at the right-most location (120) ofthe diagonal line segment (110) and extending upwards to a top of thetransposed version of the top right quadrant and all rows of pixels inthe transposed version of the top right quadrant. In furtherembodiments, a second cost function is then calculated for eachpotential line segment. Each potential line segment comprises a set ofpixels, wherein each pixel of the set of pixels has a magnitude greaterthan a cutoff magnitude. As in Case 1, the second cost function iscalculated as:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment yielding a highest value of the second costfunction identifies the location of the vertical edge (108) of theplatform (104).

Another embodiment next identifies the location of the slide verticaledge (114). Locating the slide vertical edge (114) may comprisegenerating a second set of potential line segments within the transposedversion of the top right quadrant of the digital image (100), eachpotential line segment having a slope within a range of −0.2 to 0.2(since the slide vertical edge (114) is typically more slanted than thevertical edge (108) of the platform (104)). The set of potential linesegments are located within a second window within the transposedversion of the top right quadrant of the digital image. This secondwindow may comprise columns of pixels of the transposed digital imagebeginning ten pixels above the right-most location (120) of the diagonalline segment (110) extending upwards to the top of the transposedversion of the top right quadrant and all rows of pixels in thetransposed version of the top right quadrant. Calculating a third costfunction for each potential line segment may follow. Each potential linesegment may comprise a set of pixels, where each pixel of the set ofpixels has a magnitude greater than a cutoff magnitude. The third costfunction is calculated as follows:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment yielding the highest value of the third costfunction identifies the location of the slide vertical edge (114).

As in Case 1, the transposed version of the digital image was obtainedwhen locating the vertical edge (108) of the platform (104) and theslide vertical edge (114) so that the vertical edges become horizontallines in the transposed image and hence, the algorithm looks for lineswith slopes in the range [0.1, −0.1]. It is easier to define a window of0.1 around an expected slope when searching for lines with a slope closeto zero. If searching for lines with slopes close to infinity (verticallines), then the window is difficult to define.

In some embodiments, locating the horizontal edge (106) of the platform(104) in the original (non-transposed, inverse grayscale) digital imagecomprises generating a third set of potential line segments, eachpotential line segment having a slope within a range of −0.1 to 0.1.This set of potential line segments are located within a third window,the third window comprising columns of pixels beginning at the left-mostlocation (118) of the diagonal line segment (110) and extending upwardsto the first row of the original digital image, and all rows of pixelsin the top right quadrant of the original digital image (100).

A fourth cost function for each potential line segment may then becalculated, where each potential line segment comprises a set of pixelshaving a magnitude greater than the cutoff magnitude. The fourth costfunction is calculated as follows:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment yielding the highest value of the fourth costfunction identifies the location of the horizontal edge (106) of theplatform (104).

Further, locating the slide horizontal edge (112) in the original(non-transposed, inverse grayscale) digital image follows. A fourth setof potential line segments, each having a slope within a range of −0.2to 0.2 may be generated. The fourth set of potential line segments arelocated within a fourth window. The fourth window may comprise columnsof pixels beginning ten pixels above the left-most location (118) of thediagonal line segment (110) and extending upwards to the first row ofthe original digital image, and all rows of pixels in the top rightquadrant of the original digital image (100).

A fifth cost function for each potential line segment may then becalculated. Each potential line segment comprises a set of pixels, whereeach pixel of the set of pixels has a magnitude greater than a cutoffmagnitude. The fifth cost is calculated as:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment that yields the highest value of the fifthcost function is selected, thereby identifying the slide horizontal edge(112).

After identifying the locations of the pair of horizontal edges(106,112) and the pair of vertical edges (108,114) of the of theplatform (104) and the microscope slide (102), a first and seconddistance is calculated. The value of the first distance is the shortesthorizontal distance between the vertical edge (108) of the platform(104) and the slide vertical edge (114). The value of the seconddistance is the shortest vertical distance between the horizontal edge(106) of the platform (104) and the slide horizontal edge (112). If thevalue of the first distance is greater than a first threshold maximumvalue or smaller than a first threshold minimum value then amisalignment between the microscope slide (102) and the platform (104)is detected. A misalignment is also detected if the value of the seconddistance is greater than a second threshold maximum value or smallerthan a second threshold minimum value.

In further embodiments, the length, L, of the diagonal line segment iswithin a predetermined range, for example between 200 and 400 pixels.Additionally, the value of θ may be within a predetermined range, forexample between −90° and 90°.

In other embodiments, the cutoff magnitude may comprise one or morepredetermined values. For example, a cutoff magnitude range may be [0.8,0.6, 0.5, 0.4, 0.2]. For each cutoff, pixels comprising a potential linesegment must be larger than the given cutoff. The cost function (of eachor any of the five cost functions) is calculated only for those potetialline segments satisfying this constraint.

Case 3: The reference line pattern is generalized such that it mayeither be a diagonal line or a curve.

In another embodiment, detecting a misalignment between the microscopeslide (102) and the platform upon which the microscope slide (102) isdisposed comprises first locating the reference line pattern (110) thatjoins the horizontal edge (106) and the vertical edge (108) of theplatform (104). Here, the reference line pattern (110) has apredetermined length and a parametric form. The parametric form is anequation having one or more unknown coefficients that define a shape ofthe reference line pattern. In further embodiments, identifying thelocation of the reference line pattern (110) comprises generating a setof potential reference line patterns by varying the one or more unknowncoefficients within a determined range. Each potential reference linepattern has the length and the parametric form of the reference linepattern (110) and is defined by a center pixel.

Further, the center pixel of each potential reference line pattern maybe superimposed on each pixel of a plurality of pixels comprising thetop right quadrant of the (inverse grayscale) digital image. A firstcost function is calculated for each potential reference line pattern,where each potential reference line pattern is comprised of a set ofpixels. Each pixel in this set of pixels has a magnitude. In someembodiments, the first cost function may defined as follows:cost=(mean of the magnitude of the pixels comprising the potentialreference line pattern and immediate off-diagonal pixels)/(standarddeviation of the reference line pattern and immediate off-diagonalpixels+0.001).

The immediate off-diagonal pixels are pixels either one pixel above orone pixel below each pixel in a potential reference line pattern. Theseoff-diagonal pixels are considered as in some cases the potentialreference line pattern (110) may be blurred. Considering more pixelsmakes the average/standard deviation based computation more robust.Additionally, the 0.001 in the denominator ensures that division by zerowill not occur even if all of the pixels included in the computationhave the same magnitude. Further, the reference line pattern (110) iscomprised of pixels having magnitudes approximately equal in value andhaving magnitudes higher than those of surrounding pixels. Therefore,the set of pixels comprising an optimal reference line pattern have anaverage magnitude higher than the magnitudes of surrounding pixels and anear-zero standard deviation. Thus, selecting the potential line segmentthat yields a highest value of the first cost function identifies thelocation of the reference line pattern (110).

Identifying the location of the reference line pattern (110) providestwo reference points in the digital image effective for locating thevertical edge (108) of the platform (104) and the slide vertical edge(114) (and subsequently the horizontal edge (106) of the platform (104)and the slide horizontal edge (112)). These two reference points are aleft-most location (118) and a right-most location (120) of thereference line pattern (110).

In another embodiment, the next step of the algorithm is locating thevertical edge (108) of the platform (104). A transposed version of thetop right quadrant of the digital image (100) is obtained and a firstset of potential line segments are generated such that each potentialline segment has a slope between −0.1 to 0.1. This set of potential linesegments are located within a first window of the transposed version ofthe top right quadrant of the digital image. The first window maycomprise columns of pixels beginning at the right-most location (120) ofthe reference line pattern (110) and extending upwards to the top of thetransposed version of the top right quadrant and all rows of pixels inthe transposed version of the top right quadrant. In furtherembodiments, a second cost function is then calculated for eachpotential line segment. Each potential line segment comprises a set ofpixels, wherein each pixel of the set of pixels has a magnitude greaterthan a cutoff magnitude. The second cost function is defined as:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).Again, the 0.0001 in the denominator ensures that division by zero willnot occur even if all of the pixels included in the computation have thesame magnitude. The potential line segment yielding a highest value ofthe second cost function identifies the location of the vertical edge(108) of the platform (104).

Another embodiment identifies the location of the slide vertical edge(114) next. Locating the slide vertical edge (114) may comprisegenerating a second set of potential line segments within the transposedversion of the top right quadrant of the digital image (100), eachpotential line segment having a slope within a range of −0.2 to 0.2(since the slide vertical edge (114) is typically more slanted than thevertical edge (108) of the platform (104)). The set of potential linesegments are located in a second window within the transposed version ofthe top right quadrant of the digital image. This second window maycomprise columns of pixels of the transposed digital image beginning tenpixels above the right-most location (120) of the reference line pattern(110) extending upwards to the top of the transposed version of the topright quadrant and all rows of pixels in the transposed version of thetop right quadrant. Calculating a third cost function for each potentialline segment may follow. Each potential line segment may comprise a setof pixels, where each pixel of the set of pixels has a magnitude greaterthan a cutoff magnitude. The third cost function is similarly definedas:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment yielding the highest value of the third costfunction identifies the location of the slide vertical edge (114).

As in Case 1, the transposed version of the digital image was obtainedwhen locating the vertical edge (108) of the platform (104) and theslide vertical edge (114) such that the vertical edges become horizontallines in the transposed image and hence, the algorithm looks for lineswith slopes in the range [0.1, −0.1]. It is easier to define a window of0.1 around an expected slope when searching for lines with a slope closeto zero. If searching for lines with slopes close to infinity (verticallines), then the wiindow is difficult to define.

In some embodiments, locating the horizontal edge (106) of the platform(104) in the original (non-transposed, inverse grayscale) digital imagecomprises generating a third set of potential line segments, eachpotential line segment having a slope within a range of −0.1 to 0.1. Theset of potential line segments are located within a third window, thethird window comprising columns of pixels beginning at the left-mostlocation (118) of the reference line pattern (110) and extending upwardsto the first row of the original digital image, and all rows of pixelsin the top right quadrant of the original digital image (100).

A fourth cost function for each potential line segment may then becalculated, where each potential line segment comprises a set of pixelshaving a magnitude greater than a cutoff magnitude. The fourth costfunction may be defined as:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment yielding the highest value of the fourth costfunction identifies the location of the horizontal edge (106) of theplatform (104).

Further, locating the slide horizontal edge (112) in the original(non-transposed, inverse grayscale) digital image follows. A fourth setof potential line segments, each having a slope within a range of −0.2to 0.2 may be generated. The fourth set of potential line segments arelocated within a fourth window. The fourth window may comprise columnsof pixels beginning ten pixels above the left-most location (118) of thereference line pattern (110) and extending upwards to the first row ofthe original digital image and all rows of pixels in the top rightquadrant of the original digital image (100).

A fifth cost function for each potential line segment may then becalculated. Each potential line segment comprises a set of pixels, whereeach pixel of the set of pixels has a magnitude greater than a cutoffmagnitude. The fifth cost function is defined as:Cost=(the average of the magnitudes of each pixel in a potential linesegment)/(the standard deviation of the pixels comprising the potentialline segment+0.0001).The potential line segment that yields the highest value of the fifthcost function is selected, thereby identifying the slide horizontal edge(112).

After identifying the locations of the pair of horizontal edges(106,112) and the pair of vertical edges (108,114) of the of theplatform (104) and the microscope slide (102), a first and seconddistance is calculated. The value of the first distance is the shortesthorizontal distance between the vertical edge (108) of the platform(104) and the slide vertical edge (114). The value of the seconddistance is the shortest vertical distance between the horizontal edge(106) of the platform (104) and the slide horizontal edge (112). If thevalue of the first distance is greater than a first threshold maximumvalue or smaller than a first threshold minimum value then amisalignment between the microscope slide (102) and the platform (104)is detected. A misalignment is also detected if the value of the seconddistance is greater than a second threshold maximum value or smallerthan a second threshold minimum value.

In further embodiments, the cutoff magnitude may comprise one or morepredetermined values. For example, a cutoff magnitude range may be [0.8,0.6, 0.5, 0.4, 0.2]. For each cutoff, the magnitudes of pixelscomprising a potential reference line pattern must be larger than thegiven cutoff. The cost function (for each or any of the five costfunctions) is calculated only for those potential line segmentssatisfying this constraint.

In some embodiments, the reference line pattern (110) is a straight linesegment. Other embodiments feature a curve as the reference line pattern(110). For a linear reference line pattern, the paramatric form may beof the form y=ax+b, where the coefficients a and b are varied (forexample, between −20 and 20). Knowing the range of the coefficientsenables all lines for given parameter ranges to be generated (having thesame length, L). The first cost function is then computed for allgenerated lines while varying the center, c, of each line. The potentialreference line pattern yielding a maximum first cost function is thepotential reference line pattern with optimal parameters a, b, and c.

Parametric forms of curved reference line patterns may be of the formy=ax²+bx+c and the unknown coefficients (a, b, and c) may vary within apredetermined range (e.g. −20 and 20). Similar to the linear case,knowing the range of the coefficients enables all parametric curves forgiven parameter ranges to be generated (having the same length, L). Andthe optimal reference line pattern yielding the highest first costfunction is selected, where a, b, c, and d are optimized (d is the cenerpixel of the reference line pattern).

Variation of Case 1

If runttime is a constraint, then the process of finding the location ofthe diagonal line segment (110) may be adjusted. If the length, L, ofthe diagonal line segment is known, then the mean and standard deviationfor every L×L window is found, while considering the diagonal elementsin this window. To accomplish this, a filter of size L×L may be createdwhere the elements comprising a potential diagonal line and immediateoff-diagonal elements are 1 and every other pixel in the window isassigned a value of 0. (As previously mentioned, the immediateoff-diagonal elements are either one pixel above or one pixel below apotential diagonal line.) To ensure that the sum of all pixels in thisfilter (or window) is 1, each pixel is divided by the total number of ONpixels (pixels with values >0) in the filter window.

Letting X denote the entire top right quadrant of the inverse digialimage. If the top right quadrant is convolved with the L×L filter, thenat every pixel in the filtered digital image, the mean of the (potentialdiagonal line+immediate off diagonal) pixels is obtained. Let F denotethe L×L filter. Then EX (first moment)=imfilter(X², F), where imfilterdenotes the filtering operation of filtering image X with filter F. EX2(second moment)=imfilter(X²,F), where X² denotes an image where everypixel of the image is the square of the correspondng pixel in image X.Now, variance is defined as (second moment−(square of EX)). Therefore,the variance of (the diagonal and off diagonal) pixels in each L×Lwindow of the image X is computed as: VARX=(EX2−EX*EX). The featurewhich computes (mean of the diagonal and off-diagonal pixels in each L×Lwindow)/(0.001+standard deviation ofthediagonal and off-diagonal pixelsin each L×L window), can be expressed as: EX./(0.001+sqrt(VARX)). Here,./ denotes element-wise division between the two matrices, EX and(0.001+sqrt(VARX), where sqrt denotes the element-wise square rootoperation). The off-diagonal pixels are also considered since morepixels make the average/standard deviation based computation morerobust.

The fast computation of the left-most location of the diagonal linesegment are found via EX and EX2. For a given window, a threshold showswhich significant points to consider. The slope range of allowed linemodels is known. In the top right quadrant the pixel values are in therange [0,1] and the potential line pixels are higher valued. The bestcutoff magnitude value is unknown, so different cutoff magnitudes areused, e.g. [0.8, 0.6, 0.5, 0.4, 0.2]. For a given window, with aselected cutoff magnitude, the significant pixels are found (i.e. pixelswhose magnitudes exceed the cutoff). Suppposing there are N such points.Now, pairs of two points may be randomly chosen and the line connectingthem may be identified. If the fitted line model has a slope within thepermissible range of slope values, then this line may be furtherconsidered. Otherwise the algorithm moves to the next pair of randomlychosen points.

It now must be determined how many N significant points lie in the bestfitting line connecting the two points. Suppose M pixels are within <2pixels of the fitted line. The mean magnitude of these M pixels are thencomputed and if the mean magnitude>=(0.8*cutoff), then the fitted linemay be a potential good fit. The cost function associated with apotential good fit is: (mean of the M pixel magnitudes which are <2pixels of the fitted line)/(0.0001+standard deviation of the magnitudesof these M pixels). Now, supposing for a given cutoff, there are Kfitted line models which fulfill the (mean magnitudes of the pixels thatare within <2 pixels of the fitted line, >=0.8*cutoff) constraint. Thenthat line model, out of the K models, which yields the maximum value for((mean of the M pixel magnitudes which are <2 pixels of the fittedline)/(0.0001+standard deviation of the magnitudes of these M pixels))is regarded as the best fitted line model. It is possible that nosolutions are obtained for a given cutoff, if this case arises then thenext lower cutoff value is tested and the process is repeated.

Referring to FIG. 14A, in some embodiments, an image analysis system maybe utilized for detecting, in a digital image, a misalignment between amicroscope slide (102) and a platform (104) upon which the microscopeslide (102) is disposed. In some embodiments, the image analysis systemmay comprise a slide processing station comprising the platform (104), acamera (200) disposed above the microscope slide (102) and the platform(104) such that the microscope slide (102) and the platform (104) arepositioned in a field of view of the camera, a processor (202)operatively coupled to the camera (200), and memory (201). The memory(201) may be operatively coupled to the processor (202) and configuredto store digitally-encoded instructions that, when executed by theprocessor (202), cause the processor (202) to perform operationscomprising the steps detailed in any of Case 1, 2, or 3.

Referring to FIG. 14B, in other embodiments, an image analysis systemmay be utilized for detecting, in a digital image, a misalignmentbetween a microscope slide (102) and a platform (104) upon which themicroscope slide (102) is disposed, and re-positioning the slide on theplatform. In some embodiments, the image analysis system may comprisethe platform (104) configured to receive the microscopic slide (102) anda slide alignment device (204) configured to engage the microscopicslide (102) at one or more contact points for moving the microscopicslide (102) on the platform (104). Further embodiments may feature acamera (200) disposed above the microscope slide (102) and the platform(104) (such that the microscope slide (102) and the platform (104) arepositioned in a field of view of the camera), a processor (202)operatively coupled to the camera (200) and the slide alignment device(204), and a memory (201) operatively coupled to the processor (202). Insome embodiments, the camera is configured to capture an image of themicroscope slide (102) and the platform (104) to produce a capturedimage. The captured image is processed to produce an inverse grayscaleversion (“digital image”).

The memory (201) may be operatively coupled to the processor (202) andconfigured to store digitally-encoded instructions that, when executedby the processor (202), cause the processor (202) to perform operationscomprising the steps detailed in any of Case 1, 2, or 3. In furtherembodiments, if a misalignment is detected, the processor (202) cancommand the slide alignment device (204) to engage the microscopic slide(102) at the one or more contact points, move the slide, and release theslide in another position on the platform (104).

The system can repeat the steps if a misalignment between the microscopeslide (102) and the platform (104) is detected. For example, the systemcan repeated the operations comprising capturing an image, acquiring aninverse grayscale version (“digital image”), performing the stepsdetailed in any of Case 1, 2, or 3, and moving the slide until thesystem detects that the microscopic slide (102) and platform (104) arealigned.

Slide processing systems comprising the slide alignment device (204) mayinclude a variety of stations (platforms) whose translation may besimplified by a transport mechanism, such as those described in U.S.Patent Application Publication No. 2015/0323776 entitled “SpecimenProcessing Systems and Methods For Holding Slides”, the specificationsof which are incorporated herein by reference. In some embodiments, thetransfer mechanism can include, without limitation, one or more robotichandlers or arms, X-Y-Z transport systems, conveyors, or other automatedmechanisms capable of carrying items between locations. In otherembodiments, the transfer mechanism includes one or more end effectors,grippers, suction devices, holders, clamps, or other components suitablefor gripping the slide carrier.

In other embodiments, the slide alignment device (204) may comprise amechanical handler or arm. The slide alignment device (204) may furthercomprise a gripping component for engaging the microscopic slide (102)at the one or more contact point, where the gripping component caninclude, without limitation, one or more suction devices (e.g., suctioncups, pumps, vacuum pumps, etc.), mechanical grippers (e.g., jaws,clamps, pinchers, magnets, etc.), or other retention features that, forexample, prevent dropping of the slide. For example, the grippingcomponent can include a vacuum port for which a vacuum source canprovide suction at the vacuum port via supply line that is capable ofpicking up the slide and holding the slide during further transport.Sensors (e.g., pressure sensors, air pressure sensors, light sensors,etc.) can be provided to detect the presence of the slide retained bythe gripping component.

In further embodiments, the transport mechanism can operate via a singlemotion point, such as a leadscrew connected to a motor (or a slidemechanism) which translates between a first and second position toprovide accurate placement of the microscope slide. Some implementationsrelate to a transport system including: a translating member, two ormore sample carrier retaining devices attached to the translating memberat a fixed, equal spacing between adjacent sample carrier devices, and amovement mechanism connected to the translating member to move thetranslating member and the attached sample carrier retaining device backand forth between a first and second position.

At least some embodiments include a specimen processing systemcomprising a slide ejector assembly for removing slides from a slidecarrier. The slide ejector assembly includes a slide staging device andan actuator assembly. The slide staging device includes the platform andthe slide alignment device. The actuator assembly includes a slideejector positioned to move relative to the slide carrier to transferindividual slides from the slide carrier to the platform. The slides canthus be transferred to the platform without the use of, for example,mechanical gripper or suction cup devices that pull slides from onelocation to another location.

Implementations of the sample transport systems can include any one ormore of the following features, individually or in combination. Thesample carrier can include one or more of a metal glass, ceramic orplastic (e.g. the glass microscope slide). At least one of the samplecarrier retaining devices can be moved and controlled by the translatingmember to transport the sample carrier to a specific station in a properorientation to retain or release the sample carrier to a specificlocation at a station, where the proper orientation of the samplecarrier can be achieved by rotating at least the retainer portion as thesample carrier retaining device moves towards the specific station, andwhere the proper orientation of the sample carrier can be achieved byrotating at least the sample retainer portion horizontally to a specificangle from 90 to 180 degrees. The retainer portions can include a vacuumcup, an adhesive material, an electromagnet, or mechanical deviceconfigured to hold a sample carrier.

Other non-limiting examples of said slide processing systems that may beutilized herewith are described in WO2013016035 entitled “Sampletransport systems and methods”, and U.S. Pat. No. 8,883,509 entitled“Apparatus and Method for Biological Sample Processing” and granted onNov. 11, 2014, the specifications of which are incorporated herein byreference.

FURTHER PARTICULAR EMBODIMENTS Further Embodiment 1

A method for detecting, in a digital image, a misalignment between amicroscope slide (102) and a platform (104) upon which the microscopeslide (102) is disposed, wherein the digital image is divided into fourquadrants, wherein the four quadrants are a top left quadrant, a topright quadrant, a bottom left quadrant, and a bottom right quadrant,wherein the platform (104) has a horizontal edge (106) and a verticaledge (108) wherein the microscope slide (102) has a slide horizontaledge (112) and a slide vertical edge (114) that connect to form aright-angled corner (116), wherein the method comprises: obtaining thedigital image by acquiring an inverse grayscale version of an RGB imageof the microscope slide (102) and the platform (104); (b) identifying alocation of a diagonal line segment (110) that joins the horizontal edge(106) and the vertical edge (108) of the platform (104), wherein alength of the diagonal line segment (110) is known, wherein an angle, θ,between the diagonal line segment (110) and a positive x-axis of an x-ycoordinate plane measures 135 degrees, wherein identifying the locationof the diagonal line (110) segment comprises: generating a first set ofpotential line segments, wherein an angle between each potential linesegment and the positive x-axis measures 135 degrees, wherein a lengthof each potential line segment is the length of the diagonal linesegment (110), wherein the set of potential line segments are located inthe top right quadrant of the digital image (100); and calculating afirst cost function for each potential line segment, wherein eachpotential line segment comprises a set of pixels, wherein each pixel ofthe set of pixels has a magnitude; wherein the first cost function is anaverage magnitude of the potential line segment pixels and immediate offdiagonal pixels divided by a sum of a standard deviation of thepotential line segment pixels and immediate off diagonal pixels and0.0001, wherein immediate off diagonal pixels are pixels either onepixel above or one pixel below each potential line segment, wherein thediagonal line segment is comprised of a set of pixels having magnitudesapproximately equal in value and higher than magnitudes of surroundingpixels, wherein the set of pixels comprising an optimal potential linesegment have an average magnitude higher than magnitudes of surroundingpixels and a near-zero standard deviation; and selecting the potentialline segment that yields a highest value of the first cost functionthereby identifying the location of the diagonal line segment (110),wherein identifying the location of the diagonal line segment (110)provides two reference points in the digital image effective forlocating the horizontal edge (106) of the platform (104), the verticaledge (108) of the platform (104), the slide horizontal edge (112) andthe slide vertical edge (114), wherein the two reference points are aleft-most location (118) and a right-most location (120) of the diagonalline segment (110); (c) locating the vertical edge (108) of the platform(104), comprising: (i) obtaining a transposed version of the top rightquadrant of the digital image (100); (ii) generating a second set ofpotential line segments, each potential line segment having a slopewithin a range of −0.1 to 0.1, wherein the set of potential linesegments are located within a first window of the transposed version ofthe top right quadrant of the digital image, the first window comprisingcolumns of thirty pixels beginning at the right-most location (120) ofthe diagonal line segment (110) and extending upwards, and rows ofpixels beginning from the right-most location (120) of the diagonal linesegment (110) and extending (to the right) to an end of the transposedversion of the top right quadrant of the digital image (100); (iii)calculating a second cost function for each potential line segment,wherein each potential line segment comprises a set of pixels, whereineach pixel of the set of pixels has a magnitude greater than a cutoffmagnitude, wherein the second cost function is the average of themagnitudes of each pixel divided by the sum of the standard deviation ofthe set of pixels and 0.0001; and (iv) selecting the potential linesegment that yields a highest value of the second cost function therebyidentifying the location of the vertical edge (108) of the platform(104); (d) locating the slide vertical edge (114), wherein the corner(116) of the microscope slide (102) extends beyond the diagonal linesegment (110) of the platform, wherein locating the slide vertical edge(114) comprises: (i) generating a third set of potential line segmentswithin the transposed version of the top right quadrant of the digitalimage (100), wherein each potential line segment has a slope within arange of −0.2 to 0.2, wherein the set of potential line segments arelocated within a second window within the transposed version of the topright quadrant of the digital image, the second window comprisingcolumns of pixels beginning 10 pixels above the right most location(120) of the diagonal line segment (110) and extending 250 pixels abovethe right-most location (120) of the diagonal line segment (110) and allrows of pixels in the transposed version of the top right quadrant ofthe digital image; (ii) calculating a third cost function for eachpotential line segment, wherein each potential line segment comprises aset of pixels, wherein each pixel of the set of pixels has a magnitudegreater than the cutoff magnitude, wherein the third cost function isthe average of the magnitudes of each pixel divided by the sum of thestandard deviation of the set of pixels and 0.0001; and (iii) selectingthe potential line segment that yields a highest value of the third costfunction thereby identifying the location of the slide vertical edge(114); (e) locating the horizontal edge (106) of the platform (104) inthe original digital image, comprising: (i) generating a fourth set ofpotential line segments, each potential line segment having a slopewithin a range of −0.1 to 0.1, wherein the set of potential linesegments are located within a third window, the third window comprisingrows of pixels within 30 pixels above the left-most location (118) ofthe diagonal line segment (110), and all columns of pixels in the topright quadrant of the original digital image (100); (ii) calculating afourth cost function for each potential line segment, wherein eachpotential line segment comprises a set of pixels, wherein each pixel ofthe set of pixels has a magnitude greater than the cutoff magnitude,wherein the fourth cost function is the average of the magnitudes ofeach pixel divided by the sum of the standard deviation of the set ofpixels and 0.0001; and (iii) selecting the potential line segment thatyields a highest value of the fourth cost function thereby identifyingthe location of the horizontal edge (106) of the platform (104); (f)locating the slide horizontal edge (112) in the original digital image,comprising: (i) generating a fifth set of potential line segments, eachpotential line segment having a slope within a range of −0.2 to 0.2,wherein the fifth set of potential line segments are located within afourth window, the fourth window comprising rows within 33 pixels to 250pixels above the left-most location (118) of the diagonal line segment(110), and all columns in the top right quadrant of the original digitalimage (100); (ii) calculating a fifth cost function for each potentialline segment, wherein each potential line segment comprises a set ofpixels, wherein each pixel of the set of pixels has a magnitude greaterthan the cutoff magnitude, wherein the fifth cost function is theaverage of the magnitudes of each pixel divided by the sum of thestandard deviation of the set of pixels and 0.0001; and (iii) selectingthe potential line segment that yields a highest value of the fifth costfunction thereby identifying the slide horizontal edge (112); (g)calculating a value of a first distance, wherein the value of the firstdistance is a shortest horizontal distance between the vertical edge(108) of the platform (104) and the slide vertical edge (114); and (h)calculating a value of a second distance, wherein the value of thesecond distance is a shortest vertical distance between the horizontaledge (106) of the platform (104) and the slide horizontal edge (112);wherein if the value of the first distance is greater than a firstthreshold maximum value or smaller than a first threshold minimum valuethen a misalignment between the microscope slide (102) and the platform(104) is detected, wherein if the value of the second distance isgreater than a second threshold maximum value or smaller than a secondthreshold minimum value then a misalignment between the microscope slide(102) and the platform (104) is detected.

Further Embodiment 2

The method of further embodiment 1, wherein a left-most location (118)of the diagonal line segment is located within rows 25% to 50% from atop row of the digital image and within columns 55% to 83% from aleft-most column of the top right quadrant of the digital image (100).

Further Embodiment 3

The method of either of further embodiment 1 and further embodiment 2,wherein one or more predetermined values of the cutoff magnitudes areused to determine the set of potential line segments.

Further Embodiment 4

A method for detecting, in a digital image, a misalignment between amicroscope slide (102) and a platform upon which the microscope slide(102) is disposed, wherein the digital image is divided into fourquadrants, wherein the four quadrants are a top left quadrant, a topright quadrant, a bottom left quadrant, and a bottom right quadrant,wherein the platform (104) has a horizontal edge (106) and a verticaledge (108) wherein the microscope slide (102) has a slide horizontaledge (112) and a slide vertical edge (114) that connect to form aright-angled corner (116), wherein the method comprises: (a) obtainingthe digital image by acquiring an inverse grayscale version of an RGBimage of the microscope slide (102) and the platform (104); (b)identifying a location of a diagonal line segment (110) that joins thehorizontal edge (106) and the vertical edge (108) of the platform (104),wherein the diagonal line segment (110) has a length L, wherein areference angle, θ, is measured between the diagonal line segment (110)and a positive x-axis of an x-y coordinate plane, wherein identifyingthe location of the diagonal line segment comprises: (i) generating aplurality of potential line segments each having a length equal to L anda reference angle equal to θ, wherein each potential line segment isdefined by a center pixel; (ii) superimposing the center pixel of eachpotential line segment on each pixel of a plurality of pixels comprisingthe top right quadrant of the digital image; (iii) calculating a firstcost function for each potential line segment for each pixel in the topright quadrant of the digital image (100), wherein each potential linesegment is comprised of a set of pixels, wherein each pixel of the setof pixels has a magnitude, wherein the first cost function is an averagemagnitude of the potential line segment pixels and immediate offdiagonal pixels divided by a sum of a standard deviation of thepotential line segment pixels and immediate off diagonal pixels and0.0001, wherein immediate off diagonal pixels are pixels either onepixel above or one pixel below each potential line segment, wherein thediagonal line segment is comprised of a set of pixels having magnitudesapproximately equal in value and higher than magnitudes of surroundingpixels, wherein the set of pixels comprising an optimal potential linesegment have an average magnitude higher than magnitudes of surroundingpixels and a near-zero standard deviation; and (iv) selecting thepotential line segment that yields a highest value of the first costfunction thereby identifying the location of the diagonal line segment(110), wherein identifying the location of the diagonal line segment(110) provides two reference points in the digital image effective forlocating the horizontal edge (106) of the platform (104), the verticaledge (108) of the platform (104), the slide horizontal edge (112) andthe slide vertical edge (114), wherein the two reference points are aleft-most location (118) and a right-most location (120) of the diagonalline segment; (c) locating the vertical edge (108) of the platform(104), comprising: (i) obtaining a transposed version of the top rightquadrant of the digital image (100); (ii) generating a first set ofpotential line segments, each potential line segment having a slopewithin a range of −0.1 to 0.1, wherein the set of potential linesegments are located within a first window, the first window comprisingcolumns of pixels beginning at the right-most location (120) of thediagonal line segment (110) and extending upwards to a top of thetransposed version of the top right quadrant and all rows of pixels inthe transposed version of the top right quadrant; and (iii) calculatinga second cost function for each potential line segment, wherein eachpotential line segment comprises a set of pixels, wherein each pixel ofthe set of pixels has a magnitude greater than a cutoff magnitude,wherein the second cost function is the average of the magnitudes ofeach pixel divided by the sum of the standard deviation of the set ofpixels and 0.0001; and (iv) selecting the potential line segment thatyields a highest value of the second cost function thereby identifyingthe location of the vertical edge (108) of the platform (104); (d)locating the slide vertical edge (114), wherein the corner (116) of themicroscope slide (102) extends beyond the diagonal line segment (110) ofthe platform, wherein locating the slide vertical edge (114) comprises:(i) generating a second set of potential line segments within thetransposed version of the top right quadrant of the digital image (100),wherein each potential line segment has a slope within a range of −0.2to 0.2, wherein the set of potential line segments are located within asecond window comprising columns of pixels of the transposed digitalimage beginning ten pixels above the right-most location (120) of thediagonal line segment (110) extending upwards to a top of the transposedversion of the top right quadrant and all rows of pixels in thetransposed version of the top right quadrant; and (ii) calculating athird cost function for each potential line segment, wherein eachpotential line segment comprises a set of pixels, wherein each pixel ofthe set of pixels has a magnitude greater than a cutoff magnitude,wherein the third cost function is the average of the magnitudes of eachpixel divided by the sum of the standard deviation of the set of pixelsand 0.0001; (iii) selecting the potential line segment that yields ahighest value of the third cost function thereby identifying thelocation of the slide vertical edge (114); (e) locating the horizontaledge (106) of the platform (104) in the original digital image,comprising: (i) generating a third set of potential line segments, eachpotential line segment having a slope within a range of −0.1 to 0.1,wherein the set of potential line segments are located within a thirdwindow, the third window comprising columns of pixels beginning at theleft-most location (118) of the diagonal line segment (110) andextending upwards to a first row of the original digital image, and allrows of pixels in the top right quadrant of the original digital image(100); (ii) calculating a fourth cost function for each potential linesegment, wherein each potential line segment comprises a set of pixels,wherein each pixel of the set of pixels has a magnitude greater than acutoff magnitude, wherein the fourth cost function is the average of themagnitudes of each pixel divided by the sum of the standard deviation ofthe set of pixels and 0.0001; and (iii) selecting the potential linesegment that yields a highest value of the fourth cost function therebyidentifying the location of the horizontal edge (106) of the platform(104); (f) locating the slide horizontal edge (112) in the originaldigital image, comprising: (i) generating a fourth set of potential linesegments, each potential line segment having a slope within a range of−0.2 to 0.2, wherein the set of potential line segments are locatedwithin a fourth window, the fourth window comprising columns of pixelsbeginning ten pixels above the left-most location (118) of the diagonalline segment (110) and extending upwards to a first row of the originaldigital image, and all rows of pixels in the top right quadrant of theoriginal digital image (100); (ii) calculating a fifth cost function foreach potential line segment, wherein each potential line segmentcomprises a set of pixels, wherein each pixel of the set of pixels has amagnitude greater than a cutoff magnitude, wherein the fifth costfunction is the average of the magnitudes of each pixel divided by thesum of the standard deviation of the set of pixels and 0.0001; and (iii)selecting the potential line segment that yields a highest value of thefifth cost function thereby identifying the slide horizontal edge (112);(g) calculating a value of a first distance, wherein the value of thefirst distance is a shortest horizontal distance between the verticaledge (108) of the platform (104) and the slide vertical edge (114); and(h) calculating a value of a second distance, wherein the value of thesecond distance is a shortest vertical distance between the horizontaledge (106) of the platform (104) and the slide horizontal edge (112);wherein if the value of the first distance is greater than a firstthreshold maximum value or smaller than a first threshold minimum valuethen a misalignment between the microscope slide (102) and the platform(104) is detected, wherein if the value of the second distance isgreater than a second threshold maximum value or smaller than a firstthreshold minimum value then a misalignment between the microscope slide(102) and the platform (104) is detected.

Further Embodiment 5

the method of further embodiment 4, wherein the value of L is within apredetermined range.

Further Embodiment 6

the method of further embodiment 5 or further embodiment 6, wherein thevalue of θ is within a predetermined range.

Further Embodiment 7

the method of any one of further embodiments 4, 5 or 6, wherein thecutoff magnitude comprises one or more predetermined values.

Further Embodiment 8

a method for detecting, in a digital image, a misalignment between amicroscope slide (102) and a platform upon which the microscope slide(102) is disposed, wherein the digital image is divided into fourquadrants, wherein the four quadrants are a top left quadrant, a topright quadrant, a bottom left quadrant, and a bottom right quadrant,wherein the platform (104) has a horizontal edge (106) and a verticaledge (108) wherein the microscope slide (102) has a slide horizontaledge (112) and a slide vertical edge (114) that connect to form aright-angled corner (116), the method comprises: (a) obtaining thedigital image by acquiring an inverse grayscale version of an RGB imageof the microscope slide (102) and the platform (104); (b) identifying alocation of a reference line pattern (110) that connects the horizontaledge (106) and the vertical edge (108) of the platform (104), whereinthe reference line pattern has a predetermined length and a parametricform, wherein the parametric form is an equation having one or moreunknown coefficients that define a shape of the reference line pattern,wherein identifying the location of the reference line pattern (110)comprises: (i) generating a set of potential reference line patterns byvarying the one or more unknown coefficients within a determined range,each potential reference line pattern having the length and theparametric form of the reference line pattern (110), wherein eachpotential reference line pattern is defined by a center pixel; (ii)superimposing the center pixel of each potential reference line patternon each pixel of a plurality of pixels comprising the top right quadrantof the digital image (100); (iii) calculating a first cost function foreach potential reference line pattern for each pixel in the top rightquadrant of the digital image (100), wherein each potential referenceline pattern is comprised of a set of pixels, wherein each pixel of theset of pixels has a magnitude, wherein the first cost function is anaverage magnitude of the reference line pattern pixels and immediate offdiagonal pixels divided by a sum of a standard deviation of thereference line pattern pixels and immediate off diagonal pixels and0.0001, wherein immediate off diagonal pixels are pixels either onepixel above or one pixel below each reference line pattern, wherein thereference line pattern is comprised of a set of pixels having magnitudesapproximately equal in value and higher than magnitudes of surroundingpixels, wherein the set of pixels comprising an optimal potentialreference line pattern has an average magnitude higher than magnitudesof surrounding pixels and a near-zero standard deviation; and (iv)selecting the potential reference line pattern that yields a highestvalue of the first cost function thereby identifying the location of thereference line pattern (110), wherein identifying the location of thereference line pattern (110) provides two reference points in thedigital image effective for locating the horizontal edge (106) of theplatform (104), the vertical edge (108) of the platform (104), the slidehorizontal edge (112) and the slide vertical edge (114), wherein the tworeference points are a left-most location (118) and a right-mostlocation (120) of the reference line pattern; (c) locating the verticaledge (108) of the platform (104), comprising: (i) obtaining a transposedversion of the top right quadrant of the digital image (100); (ii)generating a first set of potential line segments, each potential linesegment having a slope within a range of −0.1 to 0.1, wherein the set ofpotential line segments are located within a first window, the firstwindow comprising columns of pixels beginning at the right-most location(120) of the reference line pattern (110) and extending upwards to a topof the transposed version of the top right quadrant and all rows ofpixels in the transposed version of the top right quadrant; (iii)calculating a second cost function for each potential line segment,wherein each potential line segment comprises a set of pixels, whereineach pixel of the set of pixels has a magnitude greater than a cutoffmagnitude, wherein the second cost function is a function of an averageof the magnitudes of each pixel and a standard deviation of the set ofpixels; and (iv) selecting the potential line segment that yields ahighest value of the second cost function thereby identifying thelocation of the vertical edge (108) of the platform (104); (d)identifying a location of the slide vertical edge (114), wherein thecorner (116) of the microscope slide (102) extends beyond the referenceline pattern (110) of the platform (104), wherein locating the slidevertical edge (114) comprises: (i) generating a second set of potentialline segments within the transposed version of the top right quadrant ofthe digital image (100), wherein each potential line segment has a slopewithin a range of −0.2 to 0.2, wherein the set of potential linesegments are located within a second window comprising columns of pixelsof the transposed digital image beginning ten pixels above theright-most location (120) of the reference line pattern (110) extendingupwards to a top of the transposed version of the top right quadrant andall rows of pixels in the transposed version of the top right quadrant;and (ii) calculating a third cost function for each potential linesegment, wherein each potential line segment comprises a set of pixels,wherein each pixel of the set of pixels has a magnitude greater than thecutoff magnitude, wherein the third cost function is a function of anaverage of the magnitudes of each pixel and a standard deviation of theset of pixels; (iii) selecting the potential line segment that yields ahighest value of the third cost function thereby identifying thelocation of the slide vertical edge (114); (e) locating the horizontaledge (106) of the platform (104) in the original digital image,comprising: (i) generating a third set of potential line segments, eachpotential line segment having a slope within a range of −0.1 to 0.1,wherein the set of potential line segments are located within a thirdwindow, the third window comprising columns of pixels beginning at theleft-most location (118) of the reference line pattern (110) andextending upwards to a first row of the original digital image, and allrows of pixels in the top right quadrant of the original digital image(100); (ii) calculating a fourth cost function for each potential linesegment, wherein each potential line segment comprises a set of pixels,wherein each pixel of the set of pixels has a magnitude greater than thecutoff magnitude, wherein the fourth cost function is a function of anaverage of the magnitudes of each pixel and a standard deviation of theset of pixels; and (iii) selecting the potential line segment thatyields a highest value of the fourth cost function thereby identifyingthe location of the horizontal edge (106) of the platform (104); (f)locating the slide horizontal edge (112) in the original digital image,comprising: (i) generating a fourth set of potential line segments, eachpotential line segment having a slope within a range of −0.2 to 0.2,wherein the set of potential line segments are located within a fourthwindow, the fourth window comprising columns of pixels beginning tenpixels above the left-most location (118) of the reference line pattern(110) and extending upwards to a first row of the original digitalimage, and all rows of pixels in the top right quadrant of the originaldigital image (100); (ii) calculating a fifth cost function for eachpotential line segment, wherein each potential line segment comprises aset of pixels, wherein each pixel of the set of pixels has a magnitudegreater than the cutoff magnitude, wherein the fifth cost function is afunction of an average of the magnitudes of each pixel and a standarddeviation of the set of pixels; and (iii) selecting the potential linesegment that yields a highest value of the fifth cost function therebyidentifying the slide horizontal edge (112); (g) calculating a value ofa first distance, wherein the value of the first distance is a shortesthorizontal distance between the vertical edge (108) of the platform(104) and the slide vertical edge (114); and (h) calculating a value ofa second distance, wherein value of the second distance is a shortestvertical distance between the horizontal edge (106) of the platform(104) and the slide horizontal edge (112); wherein if the value of thefirst distance is greater than a first threshold maximum value orsmaller than a first threshold minimum value then a misalignment betweenthe microscope slide (102) and the platform (104) is detected, whereinif the value of the second distance is greater than a second thresholdmaximum value or smaller than a first threshold minimum value then amisalignment between the microscope slide (102) and the platform (104)is detected.

Further Embodiment 9

the method of further embodiment 8, wherein the reference line pattern(110) is linear having a parametric form: y=ax+b.

Further Embodiment 10

the method of either of further embodiment 8 or further embodiment 9,wherein reference line pattern (110) is a curve having a parametricform: y=ax2+bx+c.

Further Embodiment 11

an image analysis system for detecting, in a digital image, amisalignment between a microscope slide (102) and a platform (104) uponwhich the microscope slide (102) is disposed, said system comprising: aslide processing station comprising the platform (104); a camera (200)disposed above the microscope slide (102) and the platform (104) suchthat the microscope slide (102) and the platform (104) are positioned ina field of view of the camera; a processor (202) operatively coupled tothe camera (200); and a memory (201) operatively coupled to theprocessor (202), configured to store digitally-encoded instructionsthat, when executed by the processor (202), cause the processor (202) toperform operations comprising: (i) capturing an image of the microscopeslide (102) and the platform (104) upon which the microscope slide (102)is disposed with the camera (200) to produce a captured image, whereinthe platform (104) has a horizontal edge (106) and a vertical edge (108)wherein the microscope slide (102) has a slide horizontal edge (112) anda slide vertical edge (114) that connect to form a right-angled corner(116); (ii) acquiring an inverse grayscale version (“digital image”) ofthe captured image; (iii) identifying a location of a reference linepattern (110) that connects the horizontal edge (106) and the verticaledge (108) of the platform (104), wherein the reference line pattern hasa predetermined length and a parametric form, wherein the parametricform is an equation having one or more unknown coefficients that definea shape of the reference line pattern, wherein identifying the locationof the reference line pattern (110) comprises: (A) generating a set ofpotential reference line patterns by varying the one or more unknowncoefficients within a determined range, each potential reference linepattern having the length and the parametric form of the reference linepattern (110), wherein each potential reference line pattern is definedby a center pixel; (B) superimposing the center pixel of each potentialreference line pattern on each pixel of a plurality of pixels comprisingthe top right quadrant of the digital image (100); (C) calculating afirst cost function for each potential reference line pattern for eachpixel in the top right quadrant of the digital image (100), wherein eachpotential reference line pattern is comprised of a set of pixels,wherein each pixel of the set of pixels has a magnitude, wherein thefirst cost function is an average magnitude of the reference linepattern pixels and immediate off diagonal pixels divided by a sum of astandard deviation of the reference line pattern pixels and immediateoff diagonal pixels and 0.0001, wherein immediate off diagonal pixelsare pixels either one pixel above or one pixel below each reference linepattern, wherein the reference line pattern is comprised of a set ofpixels having magnitudes approximately equal in value and higher thanmagnitudes of surrounding pixels, wherein the set of pixels comprisingan optimal potential reference line pattern has an average magnitudehigher than magnitudes of surrounding pixels and a near-zero standarddeviation; and (D) selecting the potential reference line pattern thatyields a highest value of the first cost function thereby identifyingthe location of the reference line pattern (110), wherein identifyingthe location of the reference line pattern (110) provides two referencepoints in the digital image effective for locating the horizontal edge(106) of the platform (104), the vertical edge (108) of the platform(104), the slide horizontal edge (112) and the slide vertical edge(114), wherein the two reference points are a left-most location (118)and a right-most location (120) of the reference line pattern; (iv)locating the vertical edge (108) of the platform (104), comprising: (A)obtaining a transposed version of the top right quadrant of the digitalimage (100); (B) generating a first set of potential line segments, eachpotential line segment having a slope within a range of −0.1 to 0.1,wherein the set of potential line segments are located within a firstwindow, the first window comprising columns of pixels beginning at theright-most location (120) of the reference line pattern (110) andextending upwards to a top of the transposed version of the top rightquadrant and all rows of pixels in the transposed version of the topright quadrant; (C) calculating a second cost function for eachpotential line segment, wherein each potential line segment comprises aset of pixels, wherein each pixel of the set of pixels has a magnitudegreater than a cutoff magnitude, wherein the second cost function is afunction of an average of the magnitudes of each pixel and a standarddeviation of the set of pixels; and (D) selecting the potential linesegment that yields a highest value of the second cost function therebyidentifying the location of the vertical edge (108) of the platform(104); (v) identifying a location of the slide vertical edge (114),wherein the corner (116) of the microscope slide (102) extends beyondthe reference line pattern (110) of the platform (104), wherein locatingthe slide vertical edge (114) comprises: (A) generating a second set ofpotential line segments within the transposed version of the top rightquadrant of the digital image (100); wherein each potential line segmenthas a slope within a range of −0.2 to 0.2, wherein the set of potentialline segments are located within a second window comprising columns ofpixels of the transposed digital image beginning ten pixels above theright-most location (120) of the reference line pattern (110) extendingupwards to a top of the transposed version of the top right quadrant andall rows of pixels in the transposed version of the top right quadrant;and (B) calculating a third cost function for each potential linesegment, wherein each potential line segment comprises a set of pixels,wherein each pixel of the set of pixels has a magnitude greater than thecutoff magnitude, wherein the third cost function is a function of anaverage of the magnitudes of each pixel and a standard deviation of theset of pixels; and (C) selecting the potential line segment that yieldsa highest value of the third cost function thereby identifying thelocation of the slide vertical edge (114); (vi) locating the horizontaledge (106) of the platform (104) in the original digital image,comprising: (A) generating a third set of potential line segments, eachpotential line segment having a slope within a range of −0.1 to 0.1,wherein the set of potential line segments are located within a thirdwindow, the third window comprising columns of pixels beginning at theleft-most location (118) of the reference line pattern (110) andextending upwards to a first row of the original digital image, and allrows of pixels in the top right quadrant of the original digital image(100); (B) calculating a fourth cost function for each potential linesegment, wherein each potential line segment comprises a set of pixels,wherein each pixel of the set of pixels has a magnitude greater than thecutoff magnitude, wherein the fourth cost function is a function of anaverage of the magnitudes of each pixel and a standard deviation of theset of pixels; and (C) selecting the potential line segment that yieldsa highest value of the fourth cost function thereby identifying thelocation of the horizontal edge (106) of the platform (104); (vii)locating the slide horizontal edge (112) in the original digital image,comprising: (A) generating a fourth set of potential line segments, eachpotential line segment having a slope within a range of −0.2 to 0.2,wherein the set of potential line segments are located within a fourthwindow, the fourth window comprising columns of pixels beginning tenpixels above the left-most location (118) of the reference line pattern(110) and extending upwards to a first row of the original digitalimage, and all rows of pixels in the top right quadrant of the originaldigital image (100); (B) calculating a fifth cost function for eachpotential line segment, wherein each potential line segment comprises aset of pixels, wherein each pixel of the set of pixels has a magnitudegreater than the cutoff magnitude, wherein the fifth cost function is afunction of an average of the magnitudes of each pixel and a standarddeviation of the set of pixels; (C) selecting the potential line segmentthat yields a highest value of the fifth cost function therebyidentifying the slide horizontal edge (112); and (D) calculating a valueof a first distance, wherein the value of the first distance is ashortest horizontal distance between the vertical edge (108) of theplatform (104) and the slide vertical edge (114); and (viii) calculatinga value of a second distance, wherein value of the second distance is ashortest vertical distance between the horizontal edge (106) of theplatform (104) and the slide horizontal edge (112); wherein if the valueof the first distance is greater than a first threshold maximum value orsmaller than a first threshold minimum value then a misalignment betweenthe microscope slide (102) and the platform (104) is detected, whereinif the value of the second distance is greater than a second thresholdmaximum value or smaller than a first threshold minimum value then amisalignment between the microscope slide (102) and the platform (104)is detected.

Further Embodiment 12

An image analysis system for detecting, in a digital image, amisalignment between a microscope slide (102) and a platform (104) uponwhich the microscope slide (102) is disposed, said system comprising:the platform (104) configured to receive the microscopic slide (102); aslide alignment device (204) configured to engage the microscopic slide(102) at one or more contact points for moving the microscopic slide(102) on the platform (104); a camera (200) disposed above themicroscope slide (102) and the platform (104) such that the microscopeslide (102) and the platform (104) are positioned in a field of view ofthe camera; a processor (202) operatively coupled to the camera (200)and the slide alignment device (204); and a memory (201) operativelycoupled to the processor (202), configured to store digitally-encodedinstructions that, when executed by the processor (202), cause theprocessor (202) to perform operations comprising: (i) capturing an imageof the microscope slide (102) and the platform (104) upon which themicroscope slide (102) is disposed with the camera (200) to produce acaptured image, wherein the platform (104) has a horizontal edge (106)and a vertical edge (108) wherein the microscope slide (102) has a slidehorizontal edge (112) and a slide vertical edge (114) that connect toform a right-angled corner (116); (ii) acquiring an inverse grayscaleversion (“digital image”) of the captured image; (iii) identifying alocation of a reference line pattern (110) that connects the horizontaledge (106) and the vertical edge (108) of the platform (104), whereinthe reference line pattern has a predetermined length and a parametricform, wherein the parametric form is an equation having one or moreunknown coefficients that define a shape of the reference line pattern,wherein identifying the location of the reference line pattern (110)comprises: (A) generating a set of potential reference line patterns byvarying the one or more unknown coefficients within a determined range,each potential reference line pattern having the length and theparametric form of the reference line pattern (110), wherein eachpotential reference line pattern is defined by a center pixel; (B)superimposing the center pixel of each potential reference line patternon each pixel of a plurality of pixels comprising the top right quadrantof the digital image (100); (C) calculating a first cost function foreach potential reference line pattern for each pixel in the top rightquadrant of the digital image (100), wherein each potential referenceline pattern is comprised of a set of pixels, wherein each pixel of theset of pixels has a magnitude, wherein the first cost function is anaverage magnitude of the reference line pattern pixels and immediate offdiagonal pixels divided by a sum of a standard deviation of thereference line pattern pixels and immediate off diagonal pixels and0.0001, wherein immediate off diagonal pixels are pixels either onepixel above or one pixel below each reference line pattern, wherein thereference line pattern is comprised of a set of pixels having magnitudesapproximately equal in value and higher than magnitudes of surroundingpixels, wherein the set of pixels comprising an optimal potentialreference line pattern has an average magnitude higher than magnitudesof surrounding pixels and a near-zero standard deviation; and (D)selecting the potential reference line pattern that yields a highestvalue of the first cost function thereby identifying the location of thereference line pattern (110), wherein identifying the location of thereference line pattern (110) provides two reference points in thedigital image effective for locating the horizontal edge (106) of theplatform (104), the vertical edge (108) of the platform (104), the slidehorizontal edge (112) and the slide vertical edge (114), wherein the tworeference points are a left-most location (118) and a right-mostlocation (120) of the reference line pattern; (iv) locating the verticaledge (108) of the platform (104), comprising: (A) obtaining a transposedversion of the top right quadrant of the digital image (100); (B)generating a first set of potential line segments, each potential linesegment having a slope within a range of −0.1 to 0.1, wherein the set ofpotential line segments are located within a first window, the firstwindow comprising columns of pixels beginning at the right-most location(120) of the reference line pattern (110) and extending upwards to a topof the transposed version of the top right quadrant and all rows ofpixels in the transposed version of the top right quadrant; (C)calculating a second cost function for each potential line segment,wherein each potential line segment comprises a set of pixels, whereineach pixel of the set of pixels has a magnitude greater than a cutoffmagnitude, wherein the second cost function is a function of an averageof the magnitudes of each pixel and a standard deviation of the set ofpixels; and (D) selecting the potential line segment that yields ahighest value of the second cost function thereby identifying thelocation of the vertical edge (108) of the platform (104); (v)identifying a location of the slide vertical edge (114), wherein thecorner (116) of the microscope slide (102) extends beyond the referenceline pattern (110) of the platform (104), wherein locating the slidevertical edge (114) comprises: (A) generating a second set of potentialline segments within the transposed version of the top right quadrant ofthe digital image (100); wherein each potential line segment has a slopewithin a range of −0.2 to 0.2, wherein the set of potential linesegments are located within a second window comprising columns of pixelsof the transposed digital image beginning ten pixels above theright-most location (120) of the reference line pattern (110) extendingupwards to a top of the transposed version of the top right quadrant andall rows of pixels in the transposed version of the top right quadrant;and (B) calculating a third cost function for each potential linesegment, wherein each potential line segment comprises a set of pixels,wherein each pixel of the set of pixels has a magnitude greater than thecutoff magnitude, wherein the third cost function is a function of anaverage of the magnitudes of each pixel and a standard deviation of theset of pixels; and (C) selecting the potential line segment that yieldsa highest value of the third cost function thereby identifying thelocation of the slide vertical edge (114); (vi) locating the horizontaledge (106) of the platform (104) in the original digital image,comprising: (A) generating a third set of potential line segments, eachpotential line segment having a slope within a range of −0.1 to 0.1,wherein the set of potential line segments are located within a thirdwindow, the third window comprising columns of pixels beginning at theleft-most location (118) of the reference line pattern (110) andextending upwards to a first row of the original digital image, and allrows of pixels in the top right quadrant of the original digital image(100); (B) calculating a fourth cost function for each potential linesegment, wherein each potential line segment comprises a set of pixels,wherein each pixel of the set of pixels has a magnitude greater than thecutoff magnitude, wherein the fourth cost function is a function of anaverage of the magnitudes of each pixel and a standard deviation of theset of pixels; and (C) selecting the potential line segment that yieldsa highest value of the fourth cost function thereby identifying thelocation of the horizontal edge (106) of the platform (104); (vii)locating the slide horizontal edge (112) in the original digital image,comprising: (A) generating a fourth set of potential line segments, eachpotential line segment having a slope within a range of −0.2 to 0.2,wherein the set of potential line segments are located within a fourthwindow, the fourth window comprising columns of pixels beginning tenpixels above the left-most location (118) of the reference line pattern(110) and extending upwards to a first row of the original digitalimage, and all rows of pixels in the top right quadrant of the originaldigital image (100); (B) calculating a fifth cost function for eachpotential line segment, wherein each potential line segment comprises aset of pixels, wherein each pixel of the set of pixels has a magnitudegreater than the cutoff magnitude, wherein the fifth cost function is afunction of an average of the magnitudes of each pixel and a standarddeviation of the set of pixels; (C) selecting the potential line segmentthat yields a highest value of the fifth cost function therebyidentifying the slide horizontal edge (112); and (D) calculating a valueof a first distance, wherein the value of the first distance is ashortest horizontal distance between the vertical edge (108) of theplatform (104) and the slide vertical edge (114); (viii) calculating avalue of a second distance, wherein value of the second distance is ashortest vertical distance between the horizontal edge (106) of theplatform (104) and the slide horizontal edge (112), wherein if the valueof the first distance is greater than a first threshold maximum value orsmaller than a first threshold minimum value then a misalignment betweenthe microscope slide (102) and the platform (104) is detected, whereinif the value of the second distance is greater than a second thresholdmaximum value or smaller than a first threshold minimum value, then amisalignment between the microscope slide (102) and the platform (104)is detected; commanding the slide alignment device (204) to engage themicroscopic slide (102) at the one or more contact points and move theslide if the misalignment between the microscope slide (102) and theplatform (104) is detected; commanding the slide alignment device (204)to release the microscopic slide (102) on the platform (104); andrepeating (i) to (v) if the misalignment between the microscope slide(102) and the platform (104) is detected.

Further Embodiment 13

the system of further embodiment 12, wherein the slide alignment device(204) comprises a mechanical arm.

Further Embodiment 14

the system of further embodiment 12, wherein the slide alignment device(204) further comprises a gripping component for engaging themicroscopic slide (102) at the one or more contact points.

Further Embodiment 15

the system of further embodiment 14, wherein the gripping componentcomprises one or more suction cups or mechanical grippers.

As used herein, the term “about” refers to plus or minus 10% of thereferenced number.

As used here, “a,” “an,” and “the” refer to both the singular and theplural referents unless clearly indicated otherwise. Thus, for example,“a” can refer to one or more, two or more, three or more.

As used herein, the terms “vertical” and “horizontal” refer todirections that correspond to an image frame's x and y axes and are onlyan indication of the frame of reference within a given image. Thus, forexample, if an image was obtained from a different viewpoint, it ispossible that what is “vertical” and what is “horizontal” in the imagewith regard to an external frame of reference could be exchanged.

Various modifications of the invention, in addition to those describedherein, will be apparent to those skilled in the art from the foregoingdescription. For example, while the method and system are illustrated inthe context of microscope slides, it is also possible that the methodand system could be used to ensure alignment of other substrates.Furthermore, although the examples illustrate the situation where theunderlying substrate holder is smaller than the microscope slide, it isalso possible to use the method and system in circumstances wheresubstrate is smaller than the substrate holder. Such modifications arealso intended to fall within the scope of the appended claims. Eachreference cited in the present application is incorporated herein byreference in its entirety.

Although there has been shown and described illustrative embodiments ofthe disclosed method, it will be readily apparent, it will be readilyapparent to those skilled in the art that modifications may be madethereto which do not exceed the scope of the appended claims. Therefore,the scope of the invention is only to be limited by the followingclaims. In some embodiments, the figures presented in this patentapplication are drawn to scale, including the angles, ratios ofdimensions, etc. In some embodiments, the figures are representativeonly and the claims are not limited by the dimensions of the figures. Insome embodiments, descriptions of the inventions described herein usingthe phrase “comprising” includes embodiments that could be described as“consisting of”, and as such the written description requirement forclaiming one or more embodiments of the method for detecting slideplacement accuracy for medical device instruments using the phrase“consisting of” is met.

The invention claimed is:
 1. A method of detecting a misalignmentcondition between a transparent microscope slide and a platform uponwhich the transparent microscope slide is disposed, the methodcomprising: obtaining through the transparent microscope slide an imageof at least a portion of the platform, wherein the at least the portionof the platform comprises a diagonal line segment feature, and whereinthe obtained image further comprises at least a portion of thetransparent microscope slide including a right-angled corner of thetransparent microscope slide which extends beyond the diagonal linesegment feature of the platform; identifying the diagonal line segmentfeature of the platform the obtained image, wherein the identifying ofthe diagonal line segment feature of the platform comprises selectingfrom a first set prospective set of line segments a line segment torepresent the diagonal line segment feature of the platform; identifyingin the obtained image, (i) a first edge of the platform which differsdiagonal line segment feature of the platform, and (ii) a first edge ofthe transparent microscope slide, wherein the identification of thefirst edge of the platform in the obtained image is aided byidentification of a first reference point of the diagonal line segmentfeature of the platform; and calculating a value of a first distancebetween the first edge of the platform and the first edge of thetransparent microscope slide in the obtained image, wherein if the valueof the first distance lies outside a first pre-determined range ofvalues, a misalignment condition between the microscope slide and theplatform is detected.
 2. The method of claim 1, further comprising:identifying, in the obtained image, a second edge of the platform and asecond edge of the microscope slide, wherein identification of thesecond edge of the platform in the obtained image is aided byidentification of a second reference point of the diagonal line segmentfeature of the platform; and, calculating a value of a second distancethat is a shortest distance between the second edge of the platform andthe second edge of the microscope slide, wherein if the value of thefirst distance lies outside the first pre-determined range of values orif the value of the second distance lies outside a second pre-determinedrange of values, a misalignment condition between the microscope slideand the platform is detected.
 3. The method of claim 1, wherein theobtained image comprises a grayscale image.
 4. The method of claim 1,wherein the diagonal line segment feature of the platform is located inan image frame such that it forms a known angle with an x-axis of an x-ycoordinate plane of the obtained image.
 5. The method of claim 1,wherein the diagonal line segment feature of the platform is of a knownlength.
 6. The method of claim 1, wherein selecting comprises selectinga potential line segment based on a cost function.
 7. The method ofclaim 1, wherein at least one of the first edge of the platform, thesecond edge of the platform, the first edge of the microscope slide, andthe second edge of the microscope slide appears as a vertical line in animage, and wherein the method further comprises transposing at least aportion of the obtained image prior to selecting a line segment torepresent the diagonal line segment feature of the platform, the firstedge of the platform, the second edge of the platform, the first edge ofthe microscope slide and the second edge of the microscope slide.
 8. Themethod of claim 1, further comprising, in response to detection of amisalignment condition, repositioning the microscope slide on theplatform and repeating the method of claim 1 to detect if themisalignment condition is resolved.
 9. A system for determining a slidemisalignment condition between a transparent microscope slide and aplatform, comprising: a camera disposed above the transparent microscopeslide and the platform such that at least a portion of the transparentmicroscope slide and at least a portion of the platform are positionedin a field of view of the camera; and, a processor, wherein theprocessor is configured to operate according to instructions stored in amemory to control the camera to obtain an image through the transparentmicroscope slide that includes at least a portion of the transparentmicroscope slide and at least a portion of the platform and identify inthe obtained image, a diagonal line segment feature of the platform,wherein the identifying of the diagonal line segment feature of theplatform comprises selecting from a first set prospective set of linesegments a line segment to represent the diagonal line segment featureof the platform; identify in the obtained image, a first edge of theplatform and a first edge of the transparent microscope slide, whereinthe identification of the first edge of the platform in the image isaided by identification of a first reference point of the diagonal linesegment feature of the platform; and calculate a value of a firstdistance between the first edge of the platform and the first edge ofthe transparent microscope slide in the obtained image, wherein if thevalue of the first distance lies outside a first pre-determined range ofvalues the misalignment condition is detected.
 10. The system of claim9, further comprising a slide alignment device, wherein the memoryfurther stores instructions that cause the processor to control theslide alignment device to reposition the microscope slide on theplatform.
 11. The system of claim 9, wherein the platform comprises aheating and/or cooling platform of an automated slide staining device.12. The system of claim 9, wherein the diagonal line segment feature ofthe platform is detectable in the field of view of the camera.
 13. Thesystem of claim 12, wherein the diagonal line segment feature of theplatform is of known length and/or appears in a known orientation in theimage obtained by the camera.
 14. The system of claim 10, wherein theslide alignment device is controlled by the processor according toinstructions stored in the memory that cause the slide alignment deviceto place the slide onto the platform such that a right-angled corner ofthe microscope slide extends over the diagonal line segment feature ofthe platform.
 15. A system for determining a slide misalignmentcondition between a microscope slide and a platform, comprising: acamera disposed above the microscope slide and the platform such that atleast a portion of the microscope slide and at least a portion of theplatform are positioned in a field of view of the camera; and, aprocessor, wherein the processor is configured to operate according toinstructions stored in a memory to control the camera to obtain an imagethat includes at least a portion of the microscope slide and at least aportion of the platform and identify in the image, a diagonal linesegment feature of the platform, wherein the identifying of the diagonalline segment feature of the platform comprises selecting from a firstset prospective set of line segments a line segment to represent thediagonal line segment feature of the platform; identify in the image, afirst edge of the platform and a first edge of the microscope slide,wherein identification of the first edge of the platform in the image isaided by identification of a first reference point of the diagonal linesegment feature of the platform; and calculate a value of a firstdistance between the first edge of the platform and the first edge ofthe microscope slide in the image, wherein if the value of the firstdistance lies outside a first pre-determined range of values themisalignment condition is detected, wherein the diagonal line segmentfeature of the platform is of known length and/or appears in a knownorientation in the image obtained by the camera.